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ABSTRACT 


This  thesis  addresses  the  problem  of  estimating  a  random  process  from  two 
observed  signals  sampled  at  different  rates.  The  case  where  the  low-rate  observation  has 
a  higher  signal-to-noise  ratio  than  the  high-rate  observation  is  addressed.  Both  adaptive 
and  non-adaptive  filtering  techniques  are  explored.  For  the  non-adaptive  case,  a 
multirate  version  of  the  Wiener-Hopf  optimal  filter  is  used  for  estimation.  Three  forms 
of  the  filter  are  described.  It  is  shown  that  using  both  observations  with  this  filter 
achieves  a  lower  mean-squared  error  than  using  either  sequence  alone.  Furthermore,  the 
amount  of  training  data  to  solve  for  the  filter  weights  is  comparable  to  that  needed  when 
using  either  sequence  alone.  For  the  adaptive  case,  a  multirate  version  of  the  LMS 
adaptive  algorithm  is  developed.  Both  narrowband  and  broadband  interference  are 
removed  using  the  algorithm  in  an  adaptive  noise  cancellation  scheme.  The  ability  to 
remove  interference  at  the  high  rate  using  observations  taken  at  the  low  rate  without  the 
high-rate  observations  is  demonstrated. 
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EXECUTIVE  SUMMARY 


This  thesis  addresses  the  problem  of  estimating  a  random  process  using  two 
observation  sequences;  one  sequence  is  sampled  at  a  lower  rate  than  that  of  the  estimated 
process.  This  has  potential  military  applications  in  the  areas  of  satellite-based  remote 
sensing,  network-based  sensor  suites,  and  various  other  multirate  signal  and  image 
processing  applications. 

Two  new  multirate  filtering  algorithms  are  described.  The  first  multirate  filtering 
algorithm  is  based  on  the  Wiener-Hopf  least  squares  optimal  filtering  equations.  Results 
show  that  using  both  observation  sequences  with  this  filter  provides  a  lower  mean- 
squared  error  than  when  using  a  classic  Wiener-Hopf  filter  with  either  the  high-rate  or 
low-rate  observations  alone.  Additionally,  the  amount  of  training  data  needed  to 
estimate  the  filter  weights  proved  to  be  comparable  to  that  needed  when  using  either  data 
sequence  alone.  The  second  multirate  algorithm  is  adaptive  and  based  on  the  least  mean 
square  (LMS)  algorithm  of  Widrow  and  Hoff.  Results  are  presented  in  terms  of  an 
adaptive  noise  cancellation  scenario.  The  ability  to  remove  both  narrowband  and 
broadband  interference  from  a  signal  at  the  high  rate  using  low-rate  observations  alone  is 
demonstrated. 
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I.  INTRODUCTION 


Multirate  signal  processing  has  become  an  important  area  of  digital  signal 
processing  since  there  are  few  standards  that  govern  the  rate  at  which  data  is  collected 
and  sampled.  Multirate  signal  processing  is  a  rich  field  for  research,  encompassing 
everything  from  deterministic  operations  like  sampling  rate  conversion  to  statistical 
treatments  using  multiple  observations.  Some  applications  include  sampling  rate 
conversion  for  oversampling  subsystems  for  CD  or  DAT  players  [1],  and  subband  coding 
of  speech  in  digital  communications  systems  [2],  Some  statistical  research  involves 
wavelets  for  modulating  signals  to  be  transmitted  over  channels  with  unknown 
characteristics  [3], 

This  thesis  describes  research  which  is  part  of  an  overall  project  to  investigate 
methods  of  combining  information  taken  from  sensors  with  different  sampling  rates. 
Specifically,  this  thesis  addresses  the  situation  where  an  underlying  continuous-time 
signal  is  measured  by  multiple  sensors,  each  with  a  different  sampling  rate,  and  a 
different  signal-to-noise  ratio. 

The  basis  for  this  research  was  established  previously  [4].  The  purpose  of  this 
work  was  to  1)  investigate  and  validate  the  previous  theoretical  work  with  simulation 
results;  2)  extend  the  methods  using  a  least  squares  approach,  and  3)  extend  the  methods 
to  adaptive  filtering. 

The  organization  of  this  thesis  is  depicted  in  Figure  1 .  The  thesis  consists  of  four 
chapters  and  an  appendix.  Chapter  I  introduces  the  topic  and  provides  a  description  of 
the  filtering  problem.  Chapters  II  and  III  address  related  but  separate  problems.  Chapter 
II  develops  a  multirate  form  of  the  Wiener-Hopf  equation  for  optimal  filtering  using  least 
squares  methods  and  presents  results  using  this  filter.  Chapter  III  describes  an  adaptive 
multirate  filter  and  shows  results  in  the  context  of  an  adaptive  noise  cancellation 
algorithm.  Chapter  IV  presents  conclusions  and  recommendations  for  future  studies. 

The  Appendix  provides  details  about  the  various  test  data  used  in  this  work. 
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Figure  1.  Thesis  Outline  Flow  Diagram. 


2 


II.  THE  MULTIRATE  WIENER  FILTER 


This  chapter  discusses  three  (non-adaptive)  forms  of  the  optimal  multirate  filter. 
Quantitative  results  comparing  the  single-rate  and  multirate  Wiener  filters  are  given. 

The  amount  of  training  data  required  to  estimate  the  filter  coefficients  is  addressed.  In 
particular,  the  results  of  experiments  using  various  lengths  of  training  data  are  presented. 

A.  PROBLEM  DESCRIPTION 

The  problem  to  be  considered  here  is  as  follows.  Given  sensor  data  sampled  at 
different  rates  and  with  different  signal-to-noise  ratios  (SNR),  optimally  combine  the 
data  to  form  an  estimate  of  the  original  signal.  The  system-level  diagram  for  the  case  of 
two  sensors  is  depicted  in  Figure  2. 


An  underlying  continuous-time  signal  d{t )  is  monitored  by  two  sensors  which 
produce  noisy  observation  sequences.  The  noise  sequences  are  represented  by  rjx[n\  and 
r/v[m\  and  are  assumed  to  be  additive.  Different  discrete  time  indices  ‘  n  ’  and  ‘  m  ’  are 
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used  to  indicate  the  different  sampling  rates.  One  sensor  operates  at  a  sampling  rate  of  R 
kHz  while  the  other  samples  at  R/K  kHz;  the  faster  sampling  rate  is  thus  an  integer 
multiple  of  the  slower  one.  The  factor  K  is  the  ratio  between  the  two  sensor  data  rates. 
Both  data  sequences  are  fed  into  a  multirate  filter  where  they  are  used  to  jointly  produce 

an  optimal  estimate  d[n]  of  the  original  signal  d(t) .  The  estimate  is  needed  or  desired  to 
be  computed  at  the  higher  rate.  Table  1  lists  the  notation  used  throughout  this  thesis. 


Symbol 

Meaning 

x\n\ 

The  data  sequence  which  is  sampled  at  the  higher  rate. 

y[m] 

The  data  sequence  which  is  sampled  at  the  lower  rate. 

d[n\ 

The  desired  signal  to  be  estimated. 

d[n] 

The  estimate  of  the  underlying  signal  d[n] . 

P 

The  order  of  the  filter  that  operates  on  the  high-rate  data  x\n\ . 

Q 

The  order  of  the  filter  that  operates  on  the  low-rate  data  y[/«] . 

K 

The  ratio  between  the  high  and  low  sampling  rates. 

SNRhigh 

The  signal-to-noise  ratio  of  the  high-rate  data  sequence  x[»] . 

SNRlow 

The  signal-to-noise  ratio  of  the  low-rate  data  sequence  y[/n] . 

N 

The  length  of  the  high-rate  data  sequence  x[n] . 

M 

The  length  of  the  low-rate  data  sequence  y[m] . 

Is 

Notation  denoting  the  least  squares  solution  to  the  equation. 

n 

The  discrete  time  index  of  the  high-rate  data  sequence. 

m 

The  discrete  time  index  of  the  low-rate  data  sequence. 

£{.} 

The  expected  value  of  the  expression  inside  the  braces. 

MSE 

The  mean-squared  error  between  the  desired  signal  and  estimate. 

Table  1.  List  of  Notation. 
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B.  THREE  FORMS  OF  THE  FILTER 

Three  possible  forms  of  the  filter  were  considered  in  this  thesis.  These  are 
discussed  separately  below. 

1.  Direct  Form 

The  direct  form  is  the  simplest  and  the  most  basic  form  of  the  filter;  hence  the 
name.  Figure  3  illustrates  the  structure  of  this  form  of  the  optimal  multirate  filter. 


Figure  3.  Direct  Form  of  the  Multirate  Wiener  Filter. 


The  two  noisy  sensor  observation  sequences  x[/;]  and  y\m\  are  simultaneously 
fed  into  separate  time-varying  linear  filters.  The  filter  hk[n ]  operates  at  the  sampling 
rate  of  x\n\  while  gk[m]  operates  at  the  sampling  rate  of  y\m].  The  two  outputs  are 
summed  to  produce  the  estimate  d[n].  The  filters  are  to  be  chosen  to  minimize  the 

mean-squared  error  ( MSE  )  defined  as  E  |  (d[n\-d[n 

The  simultaneous  processing  of  data  by  these  two  filters  leads  to  a  periodic  time 
dependency  in  the  filter  parameters.  This  is  illustrated  in  Figure  4  and  can  be  explained 
as  follows. 
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Figure  4.  Internal  Operation  of  Multirate  Wiener  Filter. 

Recall  that  the  time  index  of  the  high-rate  filter  is  n  while  the  time  index  of  the  low-rate 
filter  is  777  .  The  estimate  d  is  to  be  produced  at  the  high  (full)  rate.  For  the  case 
illustrated  in  Figure  4,  the  order  of  the  high-rate  filter  P  is  4,  the  order  of  the  low-rate 
filter  Q  is  2,  and  the  ratio  of  sampling  rates  K  is  3.  The  filters  hk[n]  and  gk[m]  are 
positioned  at  the  corresponding  starting  points  n0  and  m0  in  their  respective  data 
sequences.  The  estimate  d[n0\  is  obtained  with  the  filters  in  these  positions.  This  is  step 
zero  in  Figure  4.  At  step  one,  d[n0  + 1]  (i.e.,  the  estimate  for  the  following  point  of  the 
high-rate  data  sequence)  is  found  by  sliding  the  high-rate  filter  hk[n]  forward  by  one 
point  while  the  low-rate  filter  remains  in  place.  This  is  repeated  for  the  estimate 
d[n0  +  2] ,  shown  in  step  two  in  Figure  4.  Finally  at  step  three,  the  estimate  d[n0  +  3]  is 
found  by  sliding  both  filters  forward  by  one  point  in  their  respective  sequences.  Note  that 
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at  this  step  the  filters  are  now  in  the  same  relative  positions  as  the  initial  step  and  the 
process  can  be  repeated.  This  continues  for  the  entire  length  of  the  available  data 
sequences.  This  pattern  for  processing  the  data  is  what  causes  the  optimal  filters  to  be 
linear  periodically  time-varying  (LPTV).  In  general,  the  filters  have  K  unique  positions 
relative  to  each  other.  Each  unique  position  contains  a  different  set  of  data  points  and 
requires  unique  sets  of  filter  coefficients  for  the  estimate;  hence  (P  +  Q)K  filter 
parameters  are  needed  to  specify  this  filter. 


a.  Derivation  of  Estimate 

The  estimate  of  the  desired  signal  can  be  written  as: 

P- 1  Q- 1 

d[n]  =  Yjhk[i]x[n  -  i ]  +  2&L/M™  ~J"\  •  O) 

i=0  /=0 


The  time  sample  n  can  be  written  as 

n  =  Km  +  k;  k  =  «(mod  K) .  (2) 

The  subscript  k  on  the  filter  weights  is  to  indicate  that  the  filters  are  periodically  time- 
varying  with  period  K.  If  the  filter  weights  used  at  the  K'h  step  are  written  as 

h*  =  |\[0]  hk[  1]  -  hk[P- l]f,  (3) 

and 

g,  =  [&[«]  &[1]  (4) 

while  the  observation  vectors  are  defined  as 

x[n]=[x[n]  x[n- 1]  •••  x[n-(P-l)]]T  (5) 

and 

y [m]=[y[m]  y[m- 1]  •••  y[m-(Q-  l)]f.  (6) 

Then  (1)  can  be  written  as 
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j[w]  =  h[x[w]  +  g£y[/n];  k  =  n(modK) . 


(V) 


6.  Least  Squares  Formulation 

Least  squares  methods  offer  a  convenient  and  data-dependent  way  to 
solve  for  the  filter  parameters.  In  order  to  pose  the  problem  in  tenns  of  least  squares 
methods,  define  the  data  vector 

d  =  [<f[n0]  d[n0  +  1]  •  •  •  c/[n0  +  (N-  l)]]r  (8) 

where  n0  is  the  initial  point  of  the  estimate  and  N  is  the  number  of  samples.  Define  the 
data  matrices  as: 


x'kJ 

y'KJ 

x  = 

xrK+1] 

Y  = 

yrK+!] 

XK+(N-i)]J 

_yr[m0+(M-l)]_ 

where  n0  and  m0  are  the  corresponding  starting  points  in  the  observations  sequences  and 
N  =  KM.  Following  the  notation  in  [6]  we  can  write 


k  =  «(modX) 


(10) 


where  the  vector  of  values  to  be  estimated  is  formed  from  the  data  vector  d  by  taking 
every  K'h  element  beginning  with  element  k  + 1.  Likewise,  Xk  is  formed  from  X  by 

Is 

taking  every  Kth  row  of  X  beginning  with  row  k  + 1 .  The  notation  =  denotes  a  least 
squares  solution  which  minimizes  the  squared  error  between  the  left  and  right  sides  of  the 
equation.  The  solution  is  well  known  and  is  given  by 

=  [X,  Y]+d,  k  =  n(modK)  (11) 


K 

g* 
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where  ‘+’  denotes  the  Moore-Penrose  pseudoinverse  of  the  matrix  [XA  Y],  The  mean- 
squared  error  is  given  by: 

MSE  =  -Yjd[n\(d[n\-d[n^  (12) 

N  n= 0 

which  after  substituting  Equation  (2)  and  (6)  can  be  written  as 

K- 1  M- 1 

MSE  -  II  d[Km  +  k]  (d[Km  +  k]~  h  Tkx[Km  +  k]~  g[y[m]  j .  (13) 

k= 0  m=0 

This  last  equation  can  be  written  in  vector  and  matrix  notation  as 

MSE  -TiXdX  -<%ht -d;rYgi)  (14) 

n=0 

where  Ak  and  Xk  are  formed  as  in  the  discussion  following  Equation  (10). 


2.  Innovations  Form 

The  innovations  form  of  the  filter,  shown  in  Figure  5,  explicitly  shows  how  the 
low-rate  observation  sequence  contributes  to  the  estimate.  The  filter  h°[n]  in  the  top 
branch  is  time-invariant  and  is  the  optimal  filter  that  would  be  used  if  the  low-rate  data 
were  not  present.  The  filter  Hk[n]  is  used  to  predict  the  low-rate  data  from  the  high-rate 
data.  The  prediction  error  sequence,  which  is  the  innovations  sequence,  is  sent  to  the 
filter  in  the  lower  branch.  Consequently,  the  output  of  the  lower  branch  constitutes  the 
additional  information  provided  by  the  low-rate  observations.  The  explicit  handling  of 
this  new  information  is  the  reason  the  name  ‘innovations’  is  used  for  this  form  of  the 
filter. 
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Figure  5 .  Innovations  Fonn  of  Multirate  Wiener  Filter. 


This  fonn  of  the  filter  can  be  derived  as  follows.  Using  some  results  from  the 
theory  of  generalized  inverses  [8],  Equation  (10)  can  be  written  in  partitioned  form  as 


*  =[X4  Y]+d„ 

k  _ 

K  x:yc: 


xx  -x;ycx 
c  dA 


(15) 


where  C  =  (I  -  XkXk)Y.  From  the  above  equation  it  can  be  seen  that,  if  H/c  is  defined  as 
H,  =  X)  Y,  then  the  high-rate  part  of  the  filter  can  be  written  as 

h*  =  h“-HAgA  (16) 

where  hA  =  XAdA  is  the  optimal  filter  when  estimating  the  data  using  only  the  high-rate 
observations.  In  a  stochastic  process  framework,  this  filter  would  not  be  a  function  of  k 
due  to  the  stationarity  of  the  data  [4],  However  in  this  least  squares  framework,  the  filter 
does  depend  on  k,  with  hA  converging  to  a  common  value  for  long  data  sequences.  The 
filter  estimate  can  now  be  written  as 


10 


d[n]  =  h  °kTx[n]  -  g[H  Tkx[n]  +  g[y  [m] 

=  hfx[n]  +  g[(y[m]-Hlx[n])  . 


(17) 


This  produces  the  form  of  the  filter  shown  in  Figure  5. 

3.  Interpolation  Form 

This  fonn  of  the  filter  avoids  the  need  for  time-varying  filters  to  process  the  data 
sequences.  A  diagram  is  shown  in  Figure  6.  In  this  form,  the  low-rate  data  is  inter¬ 
polated  to  the  high-rate.  Both  data  sequences  are  then  processed  by  time-invariant 
filters. 


Figure  6.  Interpolation  Form  of  Multirate  Wiener  Filter. 


In  this  realization,  all  of  the  branch  filters  are  time-invariant.  This  filter  works  as 
long  as  the  input  is  stationary.  This  can  be  achieved  if  the  low-rate  data  and  correspond¬ 
ing  interpolation  filter  are  bandlimited  to  ±n!K  [5].  However  the  ideal  interpolation 
filter  is  non-causal  and,  so,  a  causal  approximation  has  to  be  used  for  this  form.  This 
may  require  the  causal  approximation  filter  to  be  of  very  high  order. 

C.  COMPUTATIONAL  REQUIREMENTS 

In  comparing  the  various  realizations  of  the  optimal  multirate  filter,  a  matter  of 
concern  is  the  number  of  filter  parameters  required  as  well  as  the  number  of  operations 
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(multiplications  and  additions)  at  each  time  step.  Table  2  lists  these  quantities  for  each 
form  of  the  filter. 


Form  of  Optimal  Filter 

Filter  Parameters 

Operations  /  Unit  Time 

Direct 

(P  +  Q)K 

P  +  Q 

Innovations 

P  +  PQK+QK 

P+PQ+Q 

Interpolation 

P  +  QK 

P+QK+I 

Table  2.  Computational  Requirements  for  the  Three  Fonns  of  the  Filter. 

The  direct  form  of  the  multirate  filter,  being  the  least  computationally  expensive 
of  the  three  forms,  requires  (P  +  Q)K  coefficients.  A  corresponding  single-rate  filter 
would  require  only  P  coefficients,  which  differs  by  approximately  a  factor  of  K.  Notice 
however  that  the  number  of  operations  per  unit  time  is  P  +  Q  which  does  not  depend  on 
K.  This  means  that  the  computational  requirement  at  each  time  step  is  the  same  as  a 
filter  with  a  fixed  set  of  coefficients  (i.e.,  a  linear  time-invariant  filter). 

The  innovations  form  is  the  most  computationally  expensive  of  the  three  forms  of 
the  filter  due  to  the  prediction  filter  Ht .  There  are  PQK  more  filter  parameters  than  for 
the  interpolation  fonn  due  to  the  prediction  process.  Additionally,  the  prediction  filter 
results  in  PQ  more  operations  than  needed  using  the  direct  form.  This  form  would  not 
likely  be  used  in  practice  due  to  the  extra  computational  cost  over  the  direct  form.  This 
form  is  useful,  however,  in  analyzing  the  reduction  in  MSE  due  to  the  presence  of  the 
low-rate  observations. 

The  interpolation  form  of  the  filter  requires  the  fewest  number  of  parameters 
P  +  QK  (the  filter  g[n]  is  assumed  to  be  of  order  QK  since  the  low-rate  data  is 
interpolated  by  K  up  to  the  high  rate).  The  number  of  operations  is  thus  dependent  on 
K  and  also  depends  on  the  order  /  of  the  interpolation  filter,  which  can  become  the 
dominant  part  of  the  computation. 
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The  length  of  data  needed  in  solving  for  these  parameters  is  a  valid  practical 
concern,  and  is  addressed  in  the  next  section. 

D.  SIMULATION  RESULTS 

1.  Preliminaries 

Since  MSE  was  used  as  the  criterion  for  the  development  of  the  filters,  it  was  also 
used  to  measure  filter  perfonnance  in  this  study.  The  mean-squared  error  in  decibels 
( MSE dB),  defined  as  MSE dB  =  101og10  MSE ,  is  used  in  the  following  discussion.  The 
data  used  for  these  experiments  is  described  in  the  Appendix. 

2.  Multirate  vs.  Single-rate  Filter 

In  comparing  the  performance  of  the  multirate  and  single-rate  filters,  a  reasonable 
question  to  ask  is  “How  much  is  performance  improved  (if  at  all)  over  using  either  of  the 
data  sequences  alone?”  This  question  can  be  answered  by  comparing  the  performance  of 
the  multirate  filter  to  that  of  a  single-rate  Wiener  filter  on  either  of  the  data  sequences 
separately.  In  the  following  experiment,  the  MSE  was  calculated  while  varying  the  order 
of  the  low-rate  filter.  The  results  are  shown  in  Tables  3  and  4. 

For  the  simulations  described  here,  the  order  of  the  high-rate  filter  P  was  30  and 
the  ratio  of  sampling  rates  K  was  10.  The  SNR  of  the  high-rate  sequence  was  0  dB. 

The  SNR  of  the  low-rate  data  sequence  was  10  dB.  The  SNRs  were  obtained  by  adding 
channel  noise  to  each  of  the  observation  sequences.  The  data  were  split  into  two  sets. 

One  set  of  data,  called  the  ‘training  set,’  was  used  to  design  the  filter.  The  filter  was  then 
applied  to  the  second  set  of  data,  which  was  called  the  ‘test  set.’  The  length  of  both  the 
training  and  test  data  sequences  was  25,000  points.  The  use  of  both  a  training  and  test  set 
helps  judge  perfonnance  of  the  filter  with  different,  but  statistically  similar,  data.  The 
MSE  on  both  the  training  and  test  sets  was  recorded;  then  the  experiment  was  repeated 
using  different  realizations  of  sensor  noise.  The  results  were  averaged  over  100  trials  of 
sensor  noise. 

CaseiU:  The  order  of  the  low-rate  filter  was  taken  to  be  Q=  3.  This  means  that 
the  low-rate  filter  uses  points  covering  the  same  time  frame  as  the  high-rate  filter.  Table 
3  shows  that  using  the  high-rate  observations  alone  leads  to  an  MSE  of  approximately 
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9.8  dB  on  both  training  and  test  data.  The  low-rate  observations,  when  used  alone,  result 
in  a  MSE  of  approximately  13.5  dB  which  is  worse  by  about  3.7  dB.  When  both  data 
sets  are  used  together  however,  the  MSE  is  about  7.6  dB  which  is  about  2.2  dB  better 
than  using  the  high-rate  data  alone. 


Sensor 

Mean-Squared  Error 

Training  Set  (dB) 

Test  Set  (dB) 

High-rate 

9.84 

9.86 

Low-rate 

13.59 

13.53 

Both 

7.65 

7.64 

Table  3.  Mean-Squared  Error  for  Q  =  3  (100  trial  average). 

Case[21:  The  order  of  the  low-rate  filter  was  taken  to  be  (9  =  10.  This  means  that 
the  low-rate  filter  uses  points  covering  a  much  larger  time  frame  than  the  high-rate  filter. 
Table  4  shows  that  the  MSE  for  the  high-rate  data  alone  remains  at  approximately  9.8  dB 
as  before.  The  MSE  for  the  low-rate  data  alone  improves  (compared  to  the  previous 
experiment)  to  about  10.5  dB.  The  use  of  both  data  sets  together  in  this  experiment 


Sensor 

Mean-Squared  Error 

Training  Set  (dB) 

Test  Set  (dB) 

High-rate 

9.84 

9.86 

Low-rate 

10.67 

10.53 

Both 

5.96 

5.77 

Table  4.  Mean-Squared  Error  for  Q  =  10  (100  trial  average). 
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results  in  a  very  significant  improvement  in  performance.  Specifically  there  is  almost  a  4 
dB  reduction  in  MSE  over  using  the  high-rate  data  alone  and  about  3  dB  over  using  the 
low-rate  data  alone. 

3.  Required  Training  Data 

The  previous  section  discussed  the  computational  requirements  of  the  filter. 
Another  matter  of  practical  concern  is  the  amount  of  training  data  needed  to  design  the 
multirate  filter.  The  length  of  data  needed  to  solve  for  these  parameters  is  investigated 
here.  The  following  results  are  based  on  the  same  data  used  in  the  previous  subsection. 

For  this  experiment,  the  length  of  data  used  to  design  the  filter  was  varied  from 
500  to  25,000  points  in  steps  of  500  points.  The  other  factors  are  as  stated  above  and  are 
reproduced  here  for  continuity:  the  order  of  the  low-rate  filter  P  was  30,  the  ratio  of 
sampling  rates  K  was  10,  the  SNR  of  the  high-rate  sequence  SNRhi  h  was  0  dB,  and  the 

SNR  of  the  low-rate  sequence  SNRlow  was  10  dB.  The  error  MSE dB  on  both  the  training 

and  test  sets  were  computed  and  plotted.  The  length  of  data  required  for  training  was 
defined  to  be  the  length  at  which  the  relative  error  between  the  training  and  test  sets  came 
to  within  one  percent.  The  results  were  averaged  over  50  trials  of  sensor  noise. 


a.  Low-rate  Filter  Order  Q  =  3 

The  figure  shows  plots  of  MSE  vs.  length  of  data  set.  The  top  pair  of 
curves  result  when  using  low-rate  data  alone;  the  middle  pair  is  for  the  high-rate  data 
alone,  while  the  bottom  pair  results  when  using  both  data  sets.  For  each  case,  the  dotted 
lines  correspond  to  the  training  sets,  and  the  solid  lines  to  the  test  sets.  The  point  at 
which  the  MSE  on  the  training  and  test  sets  come  to  within  one  percent  of  each  other  is 
circled.  Using  low-rate  data  alone,  it  took  10,000  points  of  training  data  to  reach  the 
criterion  point.  The  case  of  a  low-rate  filter  order  Q  =  3  is  depicted  in  Figure  7. 
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Figure  7. 


Mean-Squared  Error  versus  Data  Length  for  Q  =  3.  TOP:  Low-rate  data  alone. 
MIDDLE:  High-rate  data  alone.  BOTTOM:  Both  data  sets. 


Using  high-rate  data  alone,  only  4,500  points  were  needed.  Interestingly,  using  both  data 
sets,  only  7,000  points  were  needed.  This  is  not  an  unreasonably  high  cost  to  design  the 
multirate  Wiener  filter. 


b.  Low-rate  Filter  Order  (9=10 

The  above  simulation  was  repeated  using  a  low-rate  filter  order  of  (9=10. 
Ligure  8  shows  the  results  graphically.  Using  low-rate  data  alone  required  over  25,000 
points  of  data.  Designing  the  filter  using  both  observation  sequences,  however,  required 
only  4,500  points  of  data.  This  turned  out  to  be  exactly  the  same  as  the  number  required 
for  using  the  high-rate  data  alone. 
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Figure  8.  Mean-Squared  Error  versus  Data  Length  for  Q  =  10.  TOP:  Low-rate  data  alone. 
MIDDLE:  High-rate  data  alone.  BOTTOM:  Both  data  sets. 


The  results  of  these  experiments  indicate  that  the  length  of  training  data  needed  to 
estimate  the  filter  coefficients  is  comparable  to  that  when  using  time-invariant  filters  on 
either  sequence  alone. 


4.  Optimizing  Filter  Tap  Weights 

As  part  of  this  research,  a  parametric  representation  of  the  filter  impulse  response 
was  sought  to  help  to  reduce  the  number  of  weights  to  compute.  This  representation 
would  exploit  possible  patterns  in  the  filter  coefficients.  It  is  the  time-variant  filters  that 
cause  the  computations  to  increase.  Therefore,  the  time-variant  behavior  of  the  filters 
was  examined  for  a  pattern  to  exploit. 
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The  FM,  AR,  and  SINE  signals  (see  Appendix)  were  used  for  this  experiment. 

For  each  of  the  signals,  the  high-rate  and  low-rate  filters  were  examined  for  patterns. 

The  time-varying  nature  of  the  filter  coefficients,  however,  proved  to  be  signal- 
dependent. 

For  example,  a  signal  of  length  L  =  27, 000  points  was  used  to  design  a  multirate 
Wiener  filter  with  two  observations  sequences  under  the  following  conditions:  P  =  100 , 

Q  =  100 ,  K  =  20 ,  SNRhigh  =  0  dB,  and  SNRlow  =10  dB.  These  conditions  yield 

(100  +  100)20  =  4000  filter  weights  (see  Table  2).  The  filter  coefficients  [/?]  and 
g k[m]  are  plotted  in  Figure  9.  Three-dimensional  plots  of  the  filter  coefficients  for  the 

FM,  AR,  and  SINE  signals  are  shown.  The  left  column  of  coefficients  is  for  the  high-rate 
filters,  while  the  right  column  holds  the  low-rate  coefficients.  The  time-varying  axis  is 
labeled  ‘  k  ’  denoting  the  cycle  of  the  filter.  The  axis  for  the  filter  index  is  labeled  ‘  n  ’  or 
‘  m  ’  depending  on  whether  it  shows  the  index  for  high-rate  or  low-rate  filter, 
respectively. 

In  analyzing  Figure  9,  we  begin  by  considering  the  high-rate,  time-variant  filters 
(specifically,  for  the  FM  signal  (Figure  9(a)),  which  resembles  a  two-dimensional  wave). 
It  seems  that  along  the  k  axis,  the  filter  values  remain  relatively  constant  with  only  minor 
fluctuations.  This  would  also  appear  to  be  the  case  for  the  high-rate  coefficients  for  the 
AR  signal  (Figure  9(c))  and  SINE  signal  (Figure  9(e)).  In  addition,  note  that  there  is  no 
general  trend  where  the  filter  coefficients  become  small  along  the  n  or  m  direction, 
except  for  the  case  of  the  AR  signal.  A  similar  analysis  applies  to  the  low-rate  filters. 
Begin  with  the  low-rate  filters  for  the  FM  signal  (Figure  9(b)).  There  is  no  constant 
behavior  along  the  time-varying  axis.  The  low-rate  coefficients  of  the  AR  signal  (Figure 
9(d))  and  SINE  signal  (Figure  9(f))  exhibit  similar  behavior. 

Two  things  are  of  note:  1)  the  constant  behavior  of  the  high-rate  filters  along  the 
time-varying  axis  seems  to  validate  the  discussion  of  the  innovations  form  of  the  filter. 
Specifically,  the  claim  that  the  converge  to  a  common  value  for  long  data  sequences  is 
supported;  and  2)  the  filters  appear  to  be  unique  for  each  type  of  signal  estimated. 
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The  foregoing  results  covered  the  non-adaptive  forms  of  the  filter.  The  next 
chapter  introduces  an  adaptive  form  of  the  multirate  filter,  and  provides  results  in  the 
context  of  adaptive  interference  cancellation. 


20 


III.  THE  MULTIRATE  LMS  FILTER 


This  chapter  presents  an  adaptive  filtering  method  useful  for  applications  where 
the  statistics  of  the  data  are  non-stationary.  The  algorithm  is  based  on  the  least  mean- 
squares  (LMS)  algorithm  of  Widrow  and  Hoff  [9].  This  application  of  the  algorithm  is  in 
the  context  of  interference  removal.  Results  are  given  for  broadband  and  narrowband 
interference  in  an  adaptive  noise  cancellation  (ANC)  scenario. 

A.  BACKGROUND 

Adaptive  filtering  is  a  wide  discipline  of  which  LMS  and  its  variants  are  a 
significant  part.  In  an  adaptive  filtering  algorithm  the  data  to  be  filtered  is  employed  to 
find  the  optimum  filter  tap  weights. 

The  basic,  single-rate  LMS  algorithm  offers  a  low  cost,  elegant  solution  to 
adaptive  filtering.  An  arbitrary  initial  set  of  filter  coefficients  is  chosen  and  updated  at 
each  time  step  n  and  new  observation.  If  the  data  is  wide-sense  stationary  and  the 
algorithm  step  size  parameter  (see  below)  is  chosen  appropriately,  the  filter  coefficients 
will  approach  the  optimal  coefficients  defined  by  the  Wiener-Hopf  equations.  The 
nature  of  this  convergence  is  discussed  in  many  places  [7,  12].  If  the  data  is  non- 
stationary  and  slowly  time-varying,  the  filter  coefficients  tend  to  ‘track’  the  optimal 
time-varying  filter  for  the  data.  In  the  basic  LMS  method  the  filter  coefficient  vector  w 
is  updated  at  each  time  step  n  according  to  the  equation  [6] 

w[n  + 1]  =  w[«  ]  +  //<?[»  ]x[n]  (18) 

where 

e[n]  =  d[n]-wT[n]x[n]  (19) 

is  the  error  found  at  time  step  n  and //  is  a  parameter  called  the  ‘step  size.’  For  suitably 
chosen  values  of  the  step  size,  the  weights  will  converge  to  a  solution  close  to  that  of  the 
optimal  (Wiener)  filter.  For  practical  purposes,  the  bounds  on  the  step  size  are  [7] 
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(20) 


0  <  ju< 


PRM 


where  P  is  the  order  of  the  filter  and  i?v[0]  is  the  value  of  the  autocorrelation  sequence 
of  the  input  signal  at  lag  zero  (i.e.,  the  signal  power). 


B.  FILTER  DESCRIPTION 

The  multirate  LMS  estimation  algorithm  extends  the  LMS  algorithm  to  the  case 
where  multiple  inputs  at  different  sampling  rates  are  available.  Accordingly  for  the 
multirate  case,  the  equations  are  slightly  more  complicated.  In  this  discussion  it  is 
assumed,  as  in  the  previous  chapter,  that  two  inputs  are  available  (see  Figure  10)  and 
these  are  to  be  used  jointly  to  estimate  the  ‘desired’  signal  d[n] . 


Figure  10.  Simplified  Diagram  of  Multirate  LMS  Filter. 


The  input  x  is  sampled  at  the  full  rate,  i.e.,  the  rate  of  the  output  estimate.  The  input  y  is 
sampled  at  a  rate  of  UK  times  the  full  rate.  Again  at  any  time  n  ,  one  can  write 
n  =  Km  +  k  for  k  =  0,1, - 1.  Therefore,  define  a  vector  of  high-rate  data  points  as 
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(21) 


x[«]  =  [x[«]  x[n- 1]  •••  x[n-(P-  l)]r 
and  a  corresponding  vector  of  low-rate  data  points  as 

y  [m\  =  [y[m\  y[m- 1]  -  y[m-(Q- l)]f.  (22) 

The  estimate  is  then  of  the  form 

d[n]  =  hTkx[n]  +  gTky[m];  n  =  Km  +  k.  (23) 

In  the  absence  of  adaptation  the  filter  coefficients  would  be  periodic,  i.e.,  at  any 
times  «,  and  n2  such  that  nx  (mod  K)  =  n2  (mod  K)  =  k  the  coefficient  vectors  are  hA  and 
gk.  For  the  multirate  LMS  algorithm  these  coefficient  vectors  are  updated  in  time.  The 
coefficient  vectors  at  any  time  n  will  be  denoted  by  h k[m]  and  g k[m\  (where 
n  =  Km  +  k).  The  estimate  is  thus  given  by 

d[n]  =  h[[/;z]x[»]  +  g[ \m\y[m\  ;  k  =  n(mod K ) .  (24) 

The  error  is  then  given  by 

e[n]  =  d[n\- d[n\.  (25) 

The  update  steps  in  the  multirate  LMS  algorithm  are  defined  by 

h/  [ m  + 1]  =  hA  [m]  +  juxe[n]x[n\  (26) 

and 

g/, [m  + 1]  =  gk [m\  +  juye[n]y\m\ .  (27) 

The  complete  algorithm  is  more  easily  specified  with  a  double  index.  It  is  listed 
in  this  way  in  Table  5  and  depicted  graphically  in  Figure  11. 
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Step 

Equation 

(a) 

x,[/]  =  \x[nQ  +  iK  +  k]  x[n0  +  iK  +  k  - 1]  •  •  •  x[n0  +  iK  +  k-P  + 1]] 

y[/'l  =  [y[m(t+i]  y[m0+i- 1]  y[m0+i-Q  + 1]] 

(b) 

^[*]=hIWx*[*]+g*[»]y[»] 

(c) 

dk[i\  =  d[n0  +  iK  +  k ] 

(d) 

ekU]  =  dk[i\-  dk[i\ 

(e) 

h*[i  +  l]  =  ht[i]  +  //Iei[z]xt[i] 

g*[*  +  l]  =  g*[»]  +  ^et[i]y[*] 

Table  5.  Listing  of  the  Multirate  LMS  algorithm. 


Figure  1 1 .  Detailed  Diagram  of  Multirate  LMS  Algorithm. 


The  equations  in  Table  5  require  some  explanation.  The  time-varying  data  filters 
have  period  K,  so  there  are  PK  weights  needed  for  the  high-rate  filter  and  QK  weights 
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needed  for  the  low-rate  filter.  For  the  high-rate  case,  if  we  define  a  P  by  K  matrix 
containing  all  high-rate  coefficients 


H[/] 


h„[/']  h ,  [/] 


(28) 


whose  columns  represent  the  filter  at  each  step  k  =  0,  1,  ... ,  K  - 1  in  its  period,  then  we 
see  that  only  one  column  of  H[z]  is  updated  for  each  point  of  the  output  sequence  that  is 
estimated  (see  Table  5).  Beginning  the  estimation  at  corresponding  points  n0  and  m0  of 
the  observation  sequences,  the  filter  h()[0]  is  found.  Then  the  input  matrix  x ,  [/]  is 
updated  and  used  to  find  h,  [0] .  The  process  continues  until  all  K  columns  of  H[z]  have 
been  updated,  at  which  time  the  cycle  repeats.  The  coefficients  for  the  kIh  step  in  the 
cycle  of  the  time-varying  filter  h;  are  therefore  updated  at  every  K'h  point  of  the  original 
sequence. 

A  similar  discussion  applies  to  the  low-rate  coefficients.  The  complete  set  of 
coefficients  are  contained  in  a  Q  by  K  matrix 


G[/] 


goW  git'1 


gx-it'] 


(29) 


A  particular  column  gk  is  then  updated  at  every  K"1  point  of  the  original  data  sequence. 


C.  SIMULATION  PRELIMINARIES 

The  multirate  LMS  filter  was  tested  in  the  context  of  adaptive  interference 
cancellation,  also  known  as  adaptive  noise  cancellation  (ANC).  Results  are  given  for 
both  narrowband  and  broadband  interference.  For  the  basic,  single-rate  LMS  ANC  the 
primary  input  contains  the  signal  plus  interference.  The  so-called  ‘reference’  input 
contains  a  correlated  version  of  the  interference  that  the  adaptive  filter  uses  to  cancel  the 
interference  in  the  primary  input. 
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The  ANC  extended  to  the  case  of  multiple  reference  inputs  is  shown  in  Figure  12. 
The  inputs  to  the  filter  now  contain  multiple  versions  of  the  interference,  sampled  at 
different  rates.  The  goal  of  the  LMS  filter  is  to  estimate  the  interference  present  in  the 
reference  input  (at  the  full  rate)  from  two  noisy  independent  observations  of  the  inter¬ 
ference.  A  case  of  some  interest  is  the  case  where  the  only  reference  signal  is  y[m ] .  In 
other  words,  the  reference  signal  is  at  a  lower  rate  compared  to  the  primary  input  ,s[/;] 

and  the  cleaned  signal  cl[n] .  In  this  case  it  may  be  able  to  perform  the  ANC  with  fewer 
samples  (in  time)  of  the  correlated  interference. 
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D. 


SIMULATION  RESULTS 


Results  for  both  narrowband  and  broadband  interference  are  presented  in  this 

section. 


1.  Narrowband  Interference 

In  this  experiment  the  interference  consisted  of  tonal  ‘noise’  added  to  recorded 
speech.  Two  sinusoidal  tones  of  frequency  4410  and  8820  Hz  were  added  to  a  segment 
of  speech.  The  first  tone  was  added  at  the  start  of  the  speech  signal,  and  the  second  tone 
added  in  the  middle  of  the  sequence.  The  original  speech  signal  and  corrupted  version 
are  shown  in  Figure  13  below. 


Figure  13. 


Plot  of  voice  signal  of  the  spoken  word  ‘hello’  used  in  narrowband  interference 
simulation,  (a)  Original  speech  signal,  (b)  Speech  signal  including  tones. 
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The  test  signal  for  this  experiment  was  a  recording  of  the  spoken  word  ‘hello’ 
with  a  prolonged  ‘o.’  The  sample  of  speech  was  two  seconds  long  and  sampled  at  22.05 
kHz.  The  waveform  is  shown  in  Figure  13(a).  Two  sinusoidal  tones  were  added  to  the 
signal.  A  tone  of  4.41  kHz  was  added  at  the  beginning  of  the  speech  segment.  At  the 
midpoint  of  the  speech  segment,  this  first  tone  was  ended  and  a  second  tone  of  2.205  kHz 
was  added  for  the  remainder  of  the  speech  signal.  The  reference  input  was  generated  by 
sending  the  interference  signal  through  a  third-order  FIR  filter  and  downsampling  the 
result  (see  Appendix).  The  corrupted  signal  is  show  in  Figure  13(b)  and  illustrates  the 
adverse  effect  of  the  interference  on  signal  quality. 

The  parameters  chosen  were  as  follows.  The  high-rate  filter  order  P  was  2,  the 
low-rate  filter  order  Q  was  2  ,  the  SNR  of  the  high-rate  sequence  SNRhi  h  was  0  dB,  and 

the  SNR  of  the  low-rate  sequence  SNRlow  was  10  dB.  The  number  of  samples  required 

for  the  filter  to  mute  the  initial  sinusoid  is  used  as  a  measure  of  speed  of  convergence 
below.  The  original  speech  signal  is  reproduced  in  Figure  14(a)  for  comparison  with  the 
following  results. 


a.  Using  Both  Observation  Signals 

The  corrupted  signal  was  filtered  using  the  parameters  described  above;  a 
sampling  rate  ratio  of  K  =  4  was  used.  The  resulting  MSE  was  -14. 15  dB.  It  took 
approximately  4,575  samples  to  mute  the  initial  sinusoid  in  this  case.  For  the  given 
sampling  rate  of  22.05  kHz,  this  corresponds  to  only  0.21  seconds.  This  is  slow  for 
channel  equalization  in  a  communications  system,  but  may  suffice  for  certain  audio 
applications.  The  filtered  signal  is  shown  in  Figure  14(b).  The  spikes  in  the  sequence  at 
samples  zero  and  25,000  are  the  points  at  which  the  sinusoids  were  added. 

b.  Using  Low-Rate  Observations  Alone 

In  this  case,  the  corrupted  signal  is  filtered  using  the  low-rate  observation 
sequence  as  the  only  reference  input.  The  low-rate  observation  sequence  has  a  sampling 
rate  of  5.5 1  kHz.  The  multirate  LMS  filter  estimates  the  interference  at  the  full  rate  of 
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22.05  kHz,  using  these  samples.  All  other  simulation  parameters  are  the  same  as  the 
previous  case.  The  filtered  signal  is  shown  in  Figure  14(c).  For  this  case,  the  filter  took 
approximately  6550  samples  (0.3  sec)  to  mute  the  initial  sinusoid.  The  resulting  MSE 
was  -12.91  dB. 

This  experiment  was  also  performed  using  a  longer  segment  of  speech 
taken  from  the  cockpit  of  a  military  jet;  similar  results  were  obtained. 


0  0.5  1  1.5  2  2.5  3  3.5  4  4.5  5 


Figure  14.  Signals  after  filtering,  (a)  Original  speech  signal,  (b)  Filtered  signal  using  both 
data  sequences  with  K  =  4.  (c)  Filtered  signal  using  low-rate  data  alone  with  K  =  4. 


2.  Broadband  Interference 

An  adaptive  interference  scenario  was  also  used  for  this  scenario.  The  desired 
signal  for  this  experiment  was  a  segment  of  radio  traffic  from  an  air  traffic  control  center. 
The  interference  was  a  spoken  phrase  of  speech,  which  is  a  broadband  signal.  The 
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reference  input  was  generated  by  sending  the  interference  signal  through  an  FIR  filter 
(see  Appendix)  and  then  downsampling  the  resulting  sequence  by  a  factor  of  K .  The 
interference  was  removed  using  the  low-rate  observations  alone.  The  parameters  for  this 
simulation  were:  the  order  of  the  low-rate  filter  Q  =  40,  the  low-rate  SNR  was 
SNRlow  =  -0.26  dB,  the  sampling  rate  ratio  was  K  =  4.  The  SNR  of  the  filtered  sequence 

was  9.65  dB,  which  is  an  improvement  of  9.91  dB.  The  supporting  plots  are  shown  in 
Figure  15.  The  original  speech  signal  and  is  shown  in  Figure  15(a)  and  exhibits  some 
clipping.  The  speech  corrupted  with  interference  is  shown  in  Figure  15(b).  The  filtered 
signal  is  shown  in  Figure  15(c).  Figures  15(d)  and  (e)  show  the  actual  and  estimated 
interference  signal,  respectively. 


The  foregoing  discussion  constitutes  the  penultimate  results  of  this  thesis.  The 
next  chapter  summarizes  the  conclusions  of  this  research  and  offers  suggestions  for  future 
work. 
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amplitude  amplitude  amplitude  amplitude  amplitude 


Figure  15.  Results  for  multirate  LMS  filtering  using  low-rate  observations  alone,  (a) 
Original  speech  signal,  (b)  Signal  plus  interference,  (c)  Filtered  signal,  (d)  Original 
interference  signal,  (e)  Estimate  of  interference. 
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IV. 


CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

This  work  developed  1)  performance  results  for  the  multirate  optimal  filter  and 
multirate  LMS  adaptive  filters,  2)  a  least  squares  approach  to  finding  the  filter 
coefficients,  and  3)  an  adaptive  fonn  of  the  filter  based  on  the  LMS  algorithm.  Both  of 
these  filters  were  used  to  solve  an  estimation  problem  in  which  multiple  independent 
observations  of  the  data  were  available.  This  thesis  focused  on  the  case  of  two 
observation  sequences. 

The  multirate  optimal  filter  is  an  extension  of  the  Wiener-Hopf  optimal  filter  for 
a  single  input  sequence.  The  use  of  multiple  observation  sequences  resulted  in  a  lower 
MSE  than  the  basic  Wiener-Hopf  filter.  For  this  non-adaptive  case,  the  advantage  of 
using  both  data  sequences  over  one  alone,  the  amount  of  training  data  needed,  and 
observations  on  optimizing  the  filter  coefficients  was  addressed. 

The  multirate  LMS  filter  is  an  adaptive  filter  based  on  the  LMS  algorithm  of 
Widrow  and  Hoff.  For  this  adaptive  case,  results  for  broadband  and  narrowband 
interference  in  an  adaptive  noise  cancellation  scenario  were  given.  Results  show  that 
both  narrowband  and  broadband  interference  may  be  removed  using  the  low-rate 
observations  alone. 

1.  Optimal  Filtering 

The  performance  of  the  multirate  Wiener  filter  was  compared  to  that  of  the 
single-rate  Wiener  filter.  Since  the  MSE  while  using  the  multirate  Wiener  filter  with 
both  observation  sequences  is  consistently  lower  than  that  when  using  either  data  set 
alone,  an  advantage  that  can  be  gained  when  including  more  than  a  single  observation 
signal. 

The  length  of  training  data  needed  to  solve  for  the  filter  parameters  was 
investigated.  The  required  length  of  training  data  was  found  to  be  comparable  to  that 
when  using  either  data  sequence  alone.  Specifically,  for  the  multirate  case,  solving  for  K 
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times  as  many  coefficients  required  a  length  of  training  data  comparable  to  that  when 
using  either  the  high  or  low-rate  sequence  alone.  In  other  words,  the  increase  in  number 
of  parameters  does  not  translate  into  an  extremely  large  amount  of  training  data. 

A  parametric  repesentation  of  the  filter  impulse  response  was  investigated  and 
seems  possible  for  high  orders  of  the  filter,  but  this  was  not  further  pursued. 

2.  Adaptive  Filtering 

The  multirate  LMS  adaptive  filter  was  shown  to  perform  well  for  adaptive 
filtering  in  the  context  of  adaptive  noise  cancellation.  The  advantage  gained  from  using 
the  filter  is  the  ability  to  estimate  a  signal  at  the  full  rate  using  observations  taken  at  a 
lower  sampling  rate.  In  an  adaptive  noise  cancellation  scenario,  the  multirate  LMS  filter 
adequately  removed  (narrowband)  sinusoidal  and  (broadband)  voice  interference  from  a 
recorded  speech  signal. 


B.  RECOMMENDATIONS  FOR  FUTURE  WORK 

The  use  of  multirate  data  and  multirate  systems  is  an  important  part  of  modem 
digital  signal  processing.  This  thesis  has  investigated  one  aspect  of  the  problem,  namely 
filtering  and  estimation,  using  a  statistical  approach.  A  number  of  specific  extensions  of 
this  work  are  possible  and  would  be  beneficial. 

Finding  a  parametric  representation  of  the  multirate  filter  impulse  response  may 
be  possible;  however,  this  would  be  a  final  research  step  prior  to  implementation.  If  the 
signals  could  be  processed  to  confonn  to  a  known  model,  then  an  optimization  scheme 
based  on  the  model  might  be  used,  as  long  as  it  does  not  add  significantly  to  the 
computational  cost. 

Other  forms  of  the  optimal  filter  could  be  investigated.  A  recursive  least  squares 
(RLS)  form  would  be  a  good  next  step  since  it  is  an  adaptive  method  based  on  least 
squares  [6].  Also,  investigation  of  lattice  forms  for  these  filters  may  give  valuable  insight 
into  the  filtering  process. 


34 


In  addition  to  these  specific  topics,  other  work  is  currently  ongoing  in  Professor 
Therrien’s  research  groups  on  other  aspects  of  statistic  multirate  problems.  The  future 
work  will  include  applications  to  detection  and  classification  as  well  as  two-dimensional 
(image)  signal  processing. 
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APPENDIX 


This  appendix  discusses  how  the  simulation  data  for  this  thesis  was  generated.  All 
simulations  were  performed  in  MATLAB.  Details  are  given  for  the  methods  used  to 
generate  both  the  observed  and  desired  signals. 

A.  GENERATION  OF  OBSERVED  SIGNALS 

Figure  16  illustrates  how  data  was  generated  for  all  simulations  of  multirate 
Wiener-Hopf  filtering. 


Figure  16.  Diagram  of  data  generation  process  for  simulation  of  multirate  Wiener-Hopf 

filtering. 

A  desired  signal  sequence  d(n  )  of  length  Ld  was  generated  (see  Section  B 

below).  The  power  of  the  desired  signal  Pd  was  calculated  either  from  the  data  or  from 

the  theoretical  formula  (see  Section  B  below).  The  noisy  observations  sequences  were 

created  as  follows:  the  high-rate  observation  sequence  x{n)  was  created  by  adding  a 

pseudorandom  Gaussian  noise  sequence  to  d{n).  This  noise  sequence  is  labeled  ‘PN 

SEQ  1  ’  in  Figure  16.  The  power  of  the  noise  sequence  was  calculated  to  ensure  that  x(n ) 
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had  the  desired  signal-to-noise  ratio  of  SNRhi  h  (see  Section  C  below).  The  low-rate 

sequence  was  generated  by  adding  a  second  (unique)  pseudorandom  noise  sequence,  to 
the  desired  signal  d{n )  (labeled  ‘PN  SEQ  2’  in  Figure  16).  The  power  of  the  noise 
sequence  was  calculated  to  ensure  that  the  sequence  had  the  desired  signal-to-noise  ratio 
of  SNRlow .  The  sequence  was  then  sent  through  a  low-pass,  anti-aliasing  filter  with  a 
(normalized)  cut-off  frequency  of  1  IK  (see  Section  D  below).  The  sequence  was  then 
decimated  by  extracting  every  K,b  point  to  form  the  low-rate  observation  sequence 
y(m). 

The  reference  signals  for  ANC  in  the  multirate  LMS  experiments  were  generated 
by  running  the  interference  through  third  order  FIR  filters  with  randomly  selected 
coefficients.  Two  separate  FIR  filters  were  created,  whose  coefficients  were  random 
numbers  uniformly  distributed  between  1  and  10.  Thus,  the  coefficients  were  different 
for  each  trial  of  the  multirate  LMS  experiment.  The  high-rate  interference  was  run 
through  one  filter  to  produce  the  high-rate  reference  signal.  The  same  interference  was 
run  through  the  other  filter  and  decimated  to  produce  the  low-rate  observations. 


B.  GENERATION  OF  THE  DESIRED  SIGNALS 


The  desired  signals  for  this  thesis  were  the  AR,  FM,  and  SINE  signals,  and  a 
recorded  speech  signal.  The  AR  signal  was  obtained  by  sending  a  pseudorandom  noise 
sequence  of  variance  crw  =  0.2  through  a  single-pole  HR  filter  with  a  denominator 
polynomial  of  l-  pz1  where  p  =  0.95  with  a  sampling  rate  of  1  Hz.  The  theoretical 


power  of  this  signal  is  PAR  =  /?° 


[6].  The  FM  signal  was  a  generated  from 


the  equation  A  cos(2;r0.04t  +  cos(0.3/))  where  A  =  1 0  with  a  sampling  rate  of  1  Hz.  The 
theoretical  power  of  this  signal  is  PFM  =  A1  j 2  .  The  SINE  signal  was  generated  from  the 
equation  f>cos(2/r0. 1/)  with  B  =  VlOO  with  a  sampling  rate  of  3.33  Hz.  The  theoretical 
power  of  this  signal  is  PSINE  =  B1  / 2  .  The  speech  signal  was  sampled  at  44. 1  kHz  and 
imported  into  MATLAB  in  .WAV  format  using  the  wavread  function.  After 
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processing,  all  relevant  signals  were  saved  in  .WAV  format  using  the  wavwrite 
function. 

C.  CALCULATION  OF  THE  SIGNAL-TO-NOISE  RATIO 

The  following  logic  and  procedure  was  used  to  produce  signals  at  the  desired 
SNR  for  these  experiments.  The  SNR  in  dB  for  any  signal  is 

■W*JS=101ogjAA|  (30) 

V  ±  noise  J 

where  the  power  of  any  signal  cp  is  given  by 

P,=  TFSkA-  (3>) 

A  n= 0 

For  a  Gaussian-distributed  (nonnal)  noise  sequence,  the  power  of  the  noise  sequence  is 
its  variance. 

A  pseudorandom  noise  sequence  was  generated  using  the  randn  function  in 
MATLAB.  The  function  produces  a  sequence  of  numbers  with  a  distribution  that  is 
approximately  distributed  as  A(0,1) ,  i.e.,  normally  distributed  with  mean  zero  and 
variance  one.  If  the  desired  SNR  is  SNRdB  and  the  signal  power  is  Psi  ,  then  the  desired 

variance  of  the  noise  is  crnolse  =  Psignai  /  10(‘w^/10^  .  The  noise  sequence  A[»]  from  the 

randn  function  was  multiplied  by  the  desired  standard  deviation  to  produce  a  noise 
sequence  with  the  correct  power 

n  M  =  (V^L)jvM-  (32) 

In  order  to  reproduce  results,  the  seed  of  the  normal  random  number  generator  was 
specified.  The  two  seeds  used  in  these  experiments  were  [362436069;  521288629] 
for  the  high-rate  data,  and  [2683551084;  3690929594]  for  the  low-rate  data. 
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D. 


CALCULATION  OF  ANTI-ALIASING  FILTER  COEFFICIENTS 


For  large  sampling  rate  ratios,  the  anti-aliasing  filter  must  have  a  sharp  cutoff  in 
order  to  prevent  attenuation  of  important  signal  information.  With  this  motivation,  the 
anti-aliasing  filter  was  designed  so  that  the  transition  bandwidth  A f  is  always  10  percent 

of  the  passband  width  /  .  The  filter  was  designed  for  a  positive  frequency  spectrum 

from  0  to  1  (normalized).  Using  these  constraints,  an  FIR  filter  was  designed  using  the 
Remez  exchange  algorithm  in  MATLAB.  The  parameters  were:  the  (normalized) 
stopband  frequency  fstop  was  1  /  K ,  the  (normalized)  passband  frequency  /  was 

(0.9 )fstop,  the  passband  ripple  was  1  dB,  and  the  stopband  attenuation  was  100  dB. 
Figures  17  and  18  list  the  coefficients  of  the  FIR  antialiasing  filters  F\ri\  used  in  this 
thesis  (for  K  =  3  and  K  =  10 ),  for  those  who  wish  to  reproduce  the  results  herein.  The 
filter  output  is  given  by 

y[«]  =  &[#-i].  (33) 

i= 0 
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F(0) 

=  -7 . 98609E-05 

F  (48) 

= 

-0.006759099 

F  ( 96)  = 

=  -0.019984998 

F  (144) 

= 

-0.000491479 

F(l) 

=  -0.000361205 

F  (49) 

= 

-0.004741643 

F  ( 97 )  = 

=  -0.026823485 

F  (145) 

= 

0.003080646 

F  (2) 

=  -0.000969318 

F  (50) 

= 

0.001674979 

F  ( 98 )  : 

=  -0.011232695 

F  (146) 

= 

0.004172252 

F(3) 

=  -0.001862713 

F(51) 

= 

0.007097811 

F  ( 99)  = 

=  0.010903051 

F  (147) 

= 

0.001910078 

F  (4) 

=  -0.002665558 

F  (52  ) 

= 

0.00659168 

F(100) 

=  0.021057542 

F  ( 148 ) 

= 

-0.001878063 

F<5) 

=  -0.002646625 

F  (53) 

= 

0.000125037 

F(101) 

=  0.012838921 

F  (149) 

= 

-0.004142419 

F(6) 

=  -0.00098967 

F  (54) 

= 

-0.006997402 

F (102) 

=  -0.004696612 

F  ( 150 ) 

= 

-0.003007156 

F<7) 

=  0.002673482 

F  (55) 

= 

-0.008433678 

F  ( 103 ) 

=  -0.016242938 

F  (151) 

= 

0.000789397 

F(8) 

=  0.007742083 

F  (56) 

= 

-0.002396799 

F  ( 104 ) 

=  -0.013197333 

F  ( 152 ) 

= 

0.004470372 

F(9) 

=  0.012550675 

F  (57) 

= 

0.00634433 

F  ( 105 ) 

=  0.000272142 

F  ( 153 ) 

= 

0.005311545 

F  (10) 

=  0.014966007 

F  (58) 

= 

0.010158306 

F  (106) 

=  0.012047163 

F  ( 154 ) 

= 

0.002656322 

F  (11) 

=  0.013517386 

F  (59) 

= 

0.005143869 

F  ( 107 ) 

=  0.012709784 

F  ( 155 ) 

= 

-0.001593048 

F  ( 12 ) 

=  0.00843905 

F  (60) 

= 

-0.005014016 

F  ( 108 ) 

=  0.002871927 

F  (156) 

= 

-0.004283003 

F  (13) 

=  0.001871753 

F(61) 

= 

-0.011632244 

F (109) 

=  -0.008357338 

F  ( 157 ) 

= 

-0.003120291 

F  (14) 

=  -0.003120291 

F  ( 62 ) 

= 

-0.008357338 

F (110) 

=  -0.011632244 

F  ( 158 ) 

= 

0.001871753 

F(15) 

=  -0.004283003 

F  (63) 

= 

0.002871927 

F  ( 111 ) 

=  -0.005014016 

F  (159) 

= 

0.00843905 

F(16) 

=  -0.001593048 

F  (64) 

= 

0.012709784 

F (112) 

=  0.005143869 

F (160) 

= 

0.013517386 

F  (17) 

=  0.002656322 

F  (65) 

= 

0.012047163 

F (113) 

=  0.010158306 

F  (161) 

= 

0.014966007 

F(18) 

=  0.005311545 

F  (66) 

= 

0.000272142 

F  ( 114 ) 

=  0.00634433 

F  (162) 

= 

0.012550675 

F(19) 

=  0.004470372 

F  (67) 

= 

-0.013197333 

F  (115) 

=  -0.002396799 

F  (163) 

= 

0.007742083 

F  (20) 

=  0.000789397 

F  (68) 

= 

-0.016242938 

F  (116) 

=  -0.008433678 

F  (164) 

= 

0.002673482 

F  (21) 

=  -0.003007156 

F  (69) 

= 

-0.004696612 

F  ( 117 ) 

=  -0.006997402 

F  (165) 

= 

-0.00098967 

F  (22) 

=  -0.004142419 

F  (70) 

= 

0.012838921 

F  ( 118 ) 

=  0.000125037 

F  (166) 

= 

-0.002646625 

F  (23) 

=  -0.001878063 

F  (71) 

= 

0.021057542 

F  (119) 

=  0.00659168 

F  (167) 

= 

-0.002665558 

F  (24) 

=  0.001910078 

F  (72) 

= 

0.010903051 

F  ( 120 ) 

=  0.007097811 

F  (168) 

= 

-0.001862713 

F  (25) 

=  0.004172252 

F  (73) 

= 

-0.011232695 

F  (121) 

=  0.001674979 

F  (169) 

= 

-0.000969318 

F  (26) 

=  0.003080646 

F  (74) 

= 

-0.026823485 

F (122) 

=  -0.004741643 

F  (170) 

= 

-0.000361205 

F  (27) 

=  -0.000491479 

F  (75) 

= 

-0.019984998 

F ( 123 ) 

=  -0.006759099 

F  ( 171 ) 

= 

-7 . 98609E-05 

F  (28) 

=  -0.003631442 

F  ( 76) 

= 

0.007554848 

F (124) 

=  -0.003019216 

F  (29) 

=  -0.003731813 

F  (77) 

= 

0.034455551 

F ( 125 ) 

=  0.00296977 

F  (30) 

=  -0.000658263 

F  (78) 

= 

0.034794005 

F  (126) 

=  0.006081272 

F  (31) 

=  0.003063166 

F  (79) 

= 

0.000359361 

F (127) 

=  0.003925625 

F  ( 32 ) 

=  0.004281348 

F  (80) 

= 

-0.047179596 

F ( 128 ) 

=  -0.001359179 

F  (33) 

=  0.001866711 

F(81) 

= 

-0.066046209 

F  (129) 

=  -0.005180696 

F  (34) 

=  -0.002242833 

F  (82) 

= 

-0.022086413 

F (130) 

=  -0.004454695 

F  (35) 

=  -0.004567326 

F  (83) 

= 

0.084055579 

F (131) 

=  -6 . 1619E-05 

F  (36) 

=  -0.003009748 

F  (84) 

= 

0.210619348 

F ( 132 ) 

=  0.00412996 

F  (37) 

=  0.001259717 

F  (85) 

= 

0.29616933 

F  ( 133 ) 

=  0.004646492 

F  (38) 

=  0.004646492 

F  (86) 

= 

0.29616933 

F  ( 134 ) 

=  0.001259717 

F  (39) 

=  0.00412996 

F  (87) 

= 

0.210619348 

F  ( 135 ) 

=  -0.003009748 

F  (40) 

=  -6 . 1619E-05 

F  (88) 

= 

0.084055579 

F (136) 

=  -0.004567326 

F  (41) 

=  -0.004454695 

F  (89) 

= 

-0.022086413 

F  ( 137 ) 

=  -0.002242833 

F  (42) 

=  -0.005180696 

F  (90) 

= 

-0.066046209 

F (138) 

=  0.001866711 

F  (43) 

=  -0.001359179 

F  ( 91) 

= 

-0.047179596 

F (139) 

=  0.004281348 

F  (44) 

=  0.003925625 

F  (92) 

= 

0.000359361 

F  ( 140 ) 

=  0.003063166 

F  (45) 

=  0.006081272 

F  (93) 

= 

0.034794005 

F (141) 

=  -0.000658263 

F  (46) 

=  0.00296977 

F  (94) 

= 

0.034455551 

F (142) 

=  -0.003731813 

F  (47) 

=  -0.003019216 

F  (95) 

= 

0.007554848 

F  ( 143 ) 

=  -0.003631442 

Figure  17.  Listing  of  Antialiasing  Filter  Coefficients  (for  A  =  172  and  K  =  3). 
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F(0) 

=  • 

-1 . 05419E-05 

F 

(48) 

= 

-0.002042278 

F 

(96)  ^ 

=  • 

-0.0011838 

F  (144) 

= 

0.00018565 

F(l) 

= 

-1 .50752E-05 

F 

(49) 

= 

-0.001943538 

F 

(97)  : 

-0.000983982 

F  (145) 

= 

0.000682242 

F<2) 

= 

-2 . 48133E-05 

F 

(50) 

= 

-0.001731552 

F 

(98)  : 

-0.000704219 

F  (146) 

= 

0.001131282 

F(3) 

= 

-3 . 75925E-05 

F 

(51) 

= 

-0.001421941 

F 

(99)  : 

-0.000366504 

F  (147) 

= 

0.001494268 

F  (4) 

= 

-5.3387  6E-05 

F 

(52) 

= 

-0.001035706 

F 

(100) 

= 

2 . 38063E-06 

F  ( 148 ) 

= 

0.001739112 

F(5) 

= 

-7 . 18891E-05 

F 

(53) 

= 

-0.000598062 

F 

(101) 

= 

0.000372948 

F  (149) 

= 

0.001842947 

F(6) 

= 

-9 . 23109E-05 

F 

(54) 

= 

-0.000136994 

F 

(102) 

= 

0.000715333 

F  ( 150 ) 

= 

0.001794228 

F  ( 7 ) 

= 

-0.000113329 

F 

(55) 

= 

0.000318405 

F 

(103) 

= 

0.001001672 

F  (151) 

= 

0.001593929 

F(8) 

= 

-0.000133018 

F 

(56) 

= 

0.00073969 

F 

(104) 

= 

0.001208359 

F  ( 152 ) 

= 

0.001255721 

F(9) 

= 

-0.000148802 

F 

(57) 

= 

0.001100825 

F 

(105) 

= 

0.001317985 

F  ( 153 ) 

= 

0.000805102 

F  (10) 

= 

-0.000157442 

F 

(58) 

= 

0.001379813 

F 

(106) 

= 

0.001320806 

F  ( 154 ) 

= 

0.00027751 

F  (11) 

= 

-0.000155078 

F 

(59) 

= 

0.001560107 

F 

(107) 

= 

0.001215613 

F  ( 155 ) 

= 

-0.00028442 

F  (12) 

= 

-0.000137334 

F 

(60) 

= 

0.001631677 

F 

(108) 

= 

0.001009935 

F (156) 

= 

-0.000834273 

F(13) 

= 

-9. 94761E-05 

F 

(61) 

= 

0.001591677 

F 

(109) 

= 

0.000719533 

F  ( 157 ) 

= 

-0.00132565 

F  (14) 

= 

-3 . 66224E-05 

F 

(62) 

= 

0.001444645 

F 

(110) 

= 

0.000367229 

F  ( 158 ) 

= 

-0.001716051 

F(15) 

= 

5 . 59829E-05 

F 

(63) 

= 

0.001202208 

F 

(111) 

= 

-1 . 88572E-05 

F  (159) 

= 

-0.001970515 

F(16) 

= 

0.000182649 

F 

(64) 

= 

0.00088232 

F 

(112) 

= 

-0.000407517 

F (160) 

= 

-0.002064715 

F  (17) 

= 

0.000346882 

F 

(65) 

= 

0.00050806 

F 

(113) 

= 

-0.000766952 

F  (161) 

= 

-0.001987226 

F(18) 

= 

0.000551021 

F 

(66) 

= 

0.000106065 

F 

(114) 

= 

-0.001067351 

F  (162) 

= 

-0.001740767 

F(19) 

= 

0.000795879 

F 

(67) 

= 

-0.000295289 

F 

(115) 

= 

-0.001283346 

F  (163) 

= 

-0.001342298 

F  (20) 

= 

0.00108042 

F 

(68) 

= 

-0.000667841 

F 

(116) 

= 

-0.001396146 

F  (164) 

= 

-0.000821908 

F  (21) 

= 

0.001401494 

F 

(69) 

= 

-0.000985613 

F 

(117) 

= 

-0.001395163 

F  (165) 

= 

-0.000220603 

F  (22) 

= 

0.001753655 

F 

(70) 

= 

-0.001226625 

F 

(118) 

= 

-0.001278998 

F  (166) 

= 

0.000412893 

F  (23) 

= 

0.002129101 

F 

(71) 

= 

-0.001374458 

F 

(119) 

= 

-0.00105569 

F  (167) 

= 

0.001026035 

F  (24) 

= 

0.00251774 

F 

(72) 

= 

-0.001419424 

F 

(120) 

= 

-0.000742202 

F  (168) 

= 

0.001566783 

F  (25) 

= 

0.002907405 

F 

(73) 

= 

-0.001359274 

F 

(121) 

= 

-0.000363162 

F  (169) 

= 

0.001987969 

F  (26) 

= 

0.003284214 

F 

(74) 

= 

-0.001199369 

F 

(122) 

= 

5 . 10378E-05 

F  (170) 

= 

0.002251373 

F  (27) 

= 

0.003633083 

F 

(75) 

= 

-0.000952311 

F 

(123) 

= 

0.000466647 

F  ( 171 ) 

= 

0.002331136 

F  (28) 

= 

0.003938362 

F 

(76) 

= 

-0.00063704 

F 

(124) 

= 

0.000849283 

F  (172) 

= 

0.002216221 

F  (29) 

= 

0.004184596 

F 

(77) 

= 

-0.000277461 

F 

(125) 

= 

0.001166747 

F  (173) 

= 

0.001911688 

F  (30) 

= 

0.004357341 

F 

(78) 

= 

9 . 92 925E-05 

F 

(126) 

= 

0.001391717 

F (174) 

= 

0.001438661 

F(31) 

= 

0.004444025 

F 

(79) 

= 

0.000464864 

F 

(127) 

= 

0.001504078 

F  (175) 

= 

0.000832934 

F  (32) 

= 

0.004434777 

F 

(80) 

= 

0.000791768 

F 

(128) 

= 

0.001492708 

F  (176) 

= 

0.000142324 

F  (33) 

= 

0.004323182 

F 

(81) 

= 

0.001055448 

F 

(129) 

= 

0.001356546 

F  (177) 

= 

-0.000577052 

F  (34) 

= 

0.004106925 

F 

(82) 

= 

0.001236123 

F 

(130) 

= 

0.001104852 

F  (178) 

= 

-0.001265281 

F  (35) 

= 

0.003788239 

F 

(83) 

= 

0.001320296 

F 

(131) 

= 

0.000756627 

F  (179) 

= 

-0.0018636 

F  (36) 

= 

0.003374144 

F 

(84) 

= 

0.001301803 

F 

(132) 

= 

0.000339207 

F  ( 180 ) 

= 

-0.002319352 

F  (37) 

= 

0.002876422 

F 

(85) 

= 

0.00118231 

F 

(133) 

= 

-0.000113849 

F  ( 181 ) 

= 

-0.002590561 

F  (38) 

= 

0.002311326 

F 

(86) 

= 

0.000971224 

F 

(134) 

= 

-0.000565429 

F  ( 182 ) 

= 

-0.00264973 

F  (39) 

= 

0.001699009 

F 

(87) 

= 

0.000685014 

F 

(135) 

= 

-0.00097788 

F  ( 183 ) 

= 

-0.002486511 

F  (40) 

= 

0.001062694 

F 

(88) 

= 

0.000345984 

F 

(136) 

= 

-0.001316119 

F  ( 184 ) 

= 

-0.002109012 

F  (41) 

= 

0.000427609 

F 

(89) 

= 

-1 . 9413E-05 

F 

(137) 

= 

-0.001550598 

F  ( 185 ) 

= 

-0.001543587 

F  (42) 

= 

-0.000180249 

F 

(90) 

= 

-0.00038259 

F 

(138) 

= 

-0.001659856 

F (186) 

= 

-0.000833088 

F  (43) 

= 

-0.000735468 

F 

(91) 

= 

-0.000715033 

F 

(139) 

= 

-0.001632456 

F  ( 187 ) 

= 

-3 . 36958E-05 

F  (44) 

= 

-0.00121462 

F 

(92) 

= 

-0.000990513 

F 

(140) 

= 

-0.001468132 

F  ( 188 ) 

= 

0.000789459 

F  (45) 

= 

-0.0015976 

F 

(93) 

= 

-0.001187154 

F 

(141) 

= 

-0.001178018 

F  (189) 

= 

0.00156753 

F  (46) 

= 

-0.001868843 

F 

(94) 

= 

-0.001289171 

F 

(142) 

= 

-0.000783949 

F (190) 

= 

0.002233647 

F  (47) 

= 

-0.002018327 

F 

(95) 

= 

-0.001288156 

F 

(143) 

= 

-0.000316853 

F  (191) 

= 

0.002728587 

Figure  18.  Listing  of  Antialiasing  Filter  Coefficients  (for  A  =  570  and  K  =  10). 
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F  (192) 

= 

0.003005954 

F  (193) 

= 

0.003036426 

F  (194) 

= 

0.002810683 

F  (195) 

= 

0.002340735 

F  (196) 

= 

0.001659503 

F (197) 

= 

0.000818627 

F  (198) 

= 

-0.000115352 

F  (199) 

= 

-0.001066159 

F  (200 ) 

= 

-0.001953928 

F  (201 ) 

= 

-0.002701808 

F  (202) 

= 

-0.003242509 

F  (203 ) 

= 

-0.003524236 

F  (204) 

= 

-0.003515494 

F  (205 ) 

= 

-0.003208334 

F  (206) 

= 

-0.002619723 

F  (207 ) 

= 

-0.001790866 

F  (208 ) 

= 

-0.000784476 

F  (209) 

= 

0.000319836 

F  (210) 

= 

0.001431725 

F  (211 ) 

= 

0.002457371 

F  (212) 

= 

0.003307246 

F  (213) 

= 

0.003903767 

F  (214) 

= 

0.004188198 

F  (215) 

= 

0.004126182 

F  (216) 

= 

0.003711427 

F  (217) 

= 

0.002967161 

F  (218) 

= 

0.001945165 

F  (219) 

= 

0.000722394 

F  (220) 

= 

-0.000604618 

F  (221) 

= 

-0.001927178 

F  (222) 

= 

-0.003133118 

F  (223 ) 

= 

-0.004116078 

F  (224) 

= 

-0.004784619 

F  (225 ) 

= 

-0.005070423 

F  (226) 

= 

-0.00493486 

F  (227) 

= 

-0.004373324 

F  (228) 

= 

-0.003416894 

F  (229) 

= 

-0.002131099 

F  (230 ) 

= 

-0.000611768 

F  (231 ) 

= 

0.001021788 

F  (232 ) 

= 

0.002635803 

F  (233 ) 

= 

0.004092641 

F  (234 ) 

= 

0.005262115 

F  (235 ) 

= 

0.006032647 

F  (236) 

= 

0.006321349 

F  (237 ) 

= 

0.006082152 

F  (238 ) 

= 

0.005311247 

F  (239) 

= 

0.004049273 

F  (240) 

= 

0.002379954 

F  (241) 

= 

0.000425134 

F  (242) 

= 

-0.001663526 

F  (243 ) 

= 

-0.003715678 

F  (244) 

= 

-0.005555448 

F  (245 ) 

= 

-0.007015673 

F  (246) 

= 

-0.007952108 

F  (247) 

= 

-0.008256471 

F  (248) 

= 

-0.007867222 

F  (249) 

= 

-0.00677712 

F  (250 ) 

= 

-0.005036805 

F  (251 ) 

= 

-0.002753927 

F  (252 ) 

= 

-8 . 7 681E-05 

F  (253 ) 

= 

0.002761048 

F  (254 ) 

= 

0.005563386 

F  (255 ) 

= 

0.008079143 

F  (256) 

= 

0.010075328 

F  (257 ) 

= 

0.011345184 

F  (258 ) 

= 

0.011726323 

F  (259) 

= 

0.011116561 

F (2  60 ) 

= 

0.009486153 

F  (2  61) 

= 

0.00688534 

F (2  62 ) 

= 

0.003446444 

F (2  63 ) 

= 

-0.000619917 

F (2  64 ) 

= 

-0.005034487 

F (2  65 ) 

= 

-0.009464469 

F  (266) 

= 

-0.013543247 

F (2  67 ) 

= 

-0.016893524 

F (2  68 ) 

= 

-0.019152406 

F  (269) 

= 

-0.019996769 

F  (270) 

= 

-0.019167165 

F  (271) 

= 

-0.01648858 

F  (272) 

= 

-0.011886527 

F  (273) 

= 

-0.005397213 

F  (274) 

= 

0.002829064 

F  (275) 

= 

0.012531739 

F  (276) 

= 

0.023350962 

F  (277) 

= 

0.034845205 

F  (278) 

= 

0.046514503 

F  (279) 

= 

0.057827931 

F  (280 ) 

= 

0.068253626 

F  (281 ) 

= 

0.077289475 

F  (282) 

= 

0.08449249 

F  (283 ) 

= 

0.089505004 

F  (284) 

= 

0.092075965 

F  (285 ) 

= 

0.092075965 

F  (286) 

= 

0.089505004 

F  (287 ) 

= 

0.08449249 
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F  (288 ) 

= 

0.077289475 

F  (289) 

= 

0.068253626 

F  (290) 

= 

0.057827931 

F  (291) 

= 

0.046514503 

F (2  92 ) 

= 

0.034845205 

F (293) 

= 

0.023350962 

F (294) 

= 

0.012531739 

F (2  95 ) 

= 

0.002829064 

F  (296) 

= 

-0.005397213 

F (2  97 ) 

= 

-0.011886527 

F (2  98 ) 

= 

-0.01648858 

F  (299) 

= 

-0.019167165 

F  (300) 

= 

-0.019996769 

F  (301) 

= 

-0.019152406 

F  (302) 

= 

-0.016893524 

F  (303) 

= 

-0.013543247 

F  (304 ) 

= 

-0.009464469 

F  (305) 

= 

-0.005034487 

F  (306) 

= 

-0.000619917 

F  (307) 

= 

0.003446444 

F  (308) 

= 

0.00688534 

F  (309) 

= 

0.009486153 

F  (310) 

= 

0.011116561 

F  (311) 

= 

0.011726323 

F (312) 

= 

0.011345184 

F  (313) 

= 

0.010075328 

F  (314 ) 

= 

0.008079143 

F (315) 

= 

0.005563386 

F (316) 

= 

0.002761048 

F  (317) 

= 

-8 . 7681E-05 

F  (318) 

= 

-0.002753927 

F  (319) 

= 

-0.005036805 

F  (320) 

= 

-0.00677712 

F  (321) 

= 

-0.007867222 

F  (322) 

= 

-0.008256471 

F (323) 

= 

-0.007952108 

F  (324 ) 

= 

-0.007015673 

F  (325) 

= 

-0.005555448 

F  (326) 

= 

-0.003715678 

F  (327) 

= 

-0.001663526 

F  (328) 

= 

0.000425134 

F  (329) 

= 

0.002379954 

F  (330) 

= 

0.004049273 

F  (331) 

= 

0.005311247 

F  (332 ) 

= 

0.006082152 

F  (333) 

= 

0.006321349 

F  (334 ) 

= 

0.006032647 

F  (335) 

= 

0.005262115 

18.  (continued). 


F  (336) 

= 

0.004092641 

F  (337) 

= 

0.002635803 

F  (338) 

= 

0.001021788 

F  (339) 

= 

-0.000611768 

F (340) 

= 

-0.002131099 

F  (341 ) 

= 

-0.003416894 

F  (342 ) 

= 

-0.004373324 

F  (343) 

= 

-0.00493486 

F  (344 ) 

= 

-0.005070423 

F (345) 

= 

-0.004784619 

F (346) 

= 

-0.004116078 

F  (347 ) 

= 

-0.003133118 

F (348) 

= 

-0.001927178 

F (349) 

= 

-0.000604618 

F  (350) 

= 

0.000722394 

F  (351) 

= 

0.001945165 

F  (352) 

= 

0.002967161 

F  (353) 

= 

0.003711427 

F  (354 ) 

= 

0.004126182 

F  (355) 

= 

0.004188198 

F  (356) 

= 

0.003903767 

F  (357) 

= 

0.003307246 

F  (358) 

= 

0.002457371 

F  (359) 

= 

0.001431725 

F (360) 

= 

0.000319836 

F  (361) 

= 

-0.000784476 

F  (362 ) 

= 

-0.001790866 

F  (363) 

= 

-0.002619723 

F  (364 ) 

= 

-0.003208334 

F (365) 

= 

-0.003515494 

F (366) 

= 

-0.003524236 

F  (367 ) 

= 

-0.003242509 

F (368) 

= 

-0.002701808 

F (369) 

= 

-0.001953928 

F  (370) 

= 

-0.001066159 

F  (371) 

= 

-0.000115352 

F  (372) 

= 

0.000818627 

F  (373) 

= 

0.001659503 

F  (374) 

= 

0.002340735 

F  (375) 

= 

0.002810683 

F  (376) 

= 

0.003036426 

F  (377) 

= 

0.003005954 

F (378) 

= 

0.002728587 

F (379) 

= 

0.002233647 

F  (380) 

= 

0.00156753 

F  (381) 

= 

0.000789459 

F  (382 ) 

= 

-3 . 36958E-05 

F  (383) 

= 

-0.000833088 
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F  (384 ) 

= 

-0.001543587 

F 

(432) 

= 

-0.001550598 

F  (385) 

= 

-0.002109012 

F 

(433) 

= 

-0.001316119 

F  (386) 

= 

-0.002486511 

F 

(434) 

= 

-0.00097788 

F  (387 ) 

= 

-0.00264973 

F 

(435) 

= 

-0.000565429 

F  (388) 

= 

-0.002590561 

F 

(436) 

= 

-0.000113849 

F  (389) 

= 

-0.002319352 

F 

(437) 

= 

0.000339207 

F (390) 

= 

-0.0018636 

F 

(438) 

= 

0.000756627 

F  (391) 

= 

-0.001265281 

F 

(439) 

= 

0.001104852 

F  (392 ) 

= 

-0.000577052 

F 

(440) 

= 

0.001356546 

F  (393) 

= 

0.000142324 

F 

(441) 

= 

0.001492708 

F  (394 ) 

= 

0.000832934 

F 

(442) 

= 

0.001504078 

F (395) 

= 

0.001438661 

F 

(443) 

= 

0.001391717 

F (396) 

= 

0.001911688 

F 

(444) 

= 

0.001166747 

F (397) 

= 

0.002216221 

F 

(445) 

= 

0.000849283 

F  (398) 

= 

0.002331136 

F 

(446) 

= 

0.000466647 

F  (399) 

= 

0.002251373 

F 

(447) 

= 

5 . 10378E-05 

F  (400) 

= 

0.001987969 

F 

(448) 

= 

-0.000363162 

F  (401) 

= 

0.001566783 

F 

(449) 

= 

-0.000742202 

F  (402) 

= 

0.001026035 

F 

(450) 

= 

-0.00105569 

F  (403) 

= 

0.000412893 

F 

(451) 

= 

-0.001278998 

F  (404) 

= 

-0.000220603 

F 

(452) 

= 

-0.001395163 

F  (405) 

= 

-0.000821908 

F 

(453) 

= 

-0.001396146 

F (406) 

= 

-0.001342298 

F 

(454) 

= 

-0.001283346 

F  (407) 

= 

-0.001740767 

F 

(455) 

= 

-0.001067351 

F  (408) 

= 

-0.001987226 

F 

(456) 

= 

-0.000766952 

F  (409) 

= 

-0.002064715 

F 

(457) 

= 

-0.000407517 

F  (410) 

= 

-0.001970515 

F 

(458) 

= 

-1 . 88572E-05 

F  (411) 

= 

-0.001716051 

F 

(459) 

= 

0.000367229 

F  (412) 

= 

-0.00132565 

F 

(460) 

= 

0.000719533 

F  (413) 

= 

-0.000834273 

F 

(461) 

= 

0.001009935 

F  (414) 

= 

-0.00028442 

F 

(462) 

= 

0.001215613 

F  (415) 

= 

0.00027751 

F 

(463) 

= 

0.001320806 

F  (416) 

= 

0.000805102 

F 

(464) 

= 

0.001317985 

F  (417) 

= 

0.001255721 

F 

(465) 

= 

0.001208359 

F  (418) 

= 

0.001593929 

F 

(466) 

= 

0.001001672 

F  (419) 

= 

0.001794228 

F 

(467) 

= 

0.000715333 

F  (420) 

= 

0.001842947 

F 

(468) 

= 

0.000372948 

F  (421) 

= 

0.001739112 

F 

(469) 

= 

2 . 38063E-06 

F  (422) 

= 

0.001494268 

F 

(470) 

= 

-0.000366504 

F  (423) 

= 

0.001131282 

F 

(471) 

= 

-0.000704219 

F  (424) 

= 

0.000682242 

F 

(472) 

= 

-0.000983982 

F  (425) 

= 

0.00018565 

F 

(473) 

= 

-0.0011838 

F  (426) 

= 

-0.000316853 

F 

(474) 

= 

-0.001288156 

F  (427) 

= 

-0.000783949 

F 

(475) 

= 

-0.001289171 

F  (428) 

= 

-0.001178018 

F 

(476) 

= 

-0.001187154 

F  (429) 

= 

-0.001468132 

F 

(477) 

= 

-0.000990513 

F  (430) 

= 

-0.001632456 

F 

(478) 

= 

-0.000715033 

F  (431) 

= 

-0.001659856 

F 

(479) 

= 

-0.00038259 
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F  (480) 

= 

-1 . 9413E-05 

F 

(528) 

= 

0.000427609 

F  (481 ) 

= 

0.000345984 

F 

(529) 

= 

0.001062694 

F  (482) 

= 

0.000685014 

F 

(530) 

= 

0.001699009 

F  (483) 

= 

0.000971224 

F 

(531) 

= 

0.002311326 

F  (484) 

= 

0.00118231 

F 

(532) 

= 

0.002876422 

F  (485) 

= 

0.001301803 

F 

(533) 

= 

0.003374144 

F (486) 

= 

0.001320296 

F 

(534) 

= 

0.003788239 

F  (487) 

= 

0.001236123 

F 

(535) 

= 

0.004106925 

F  (488) 

= 

0.001055448 

F 

(536) 

= 

0.004323182 

F  (489) 

= 

0.000791768 

F 

(537) 

= 

0.004434777 

F  (490) 

= 

0.000464864 

F 

(538) 

= 

0.004444025 

F  (491) 

= 

9 . 92  925E-05 

F 

(539) 

= 

0.004357341 

F  (492) 

= 

-0.000277461 

F 

(540) 

= 

0.004184596 

F  (493) 

= 

-0.00063704 

F 

(541) 

= 

0.003938362 

F  (494) 

= 

-0.000952311 

F 

(542) 

= 

0.003633083 

F  (495) 

= 

-0.001199369 

F 

(543) 

= 

0.003284214 

F  (496) 

= 

-0.001359274 

F 

(544) 

= 

0.002907405 

F  (497) 

= 

-0.001419424 

F 

(545) 

= 

0.00251774 

F  (498) 

= 

-0.001374458 

F 

(546) 

= 

0.002129101 

F  (499) 

= 

-0.001226625 

F 

(547) 

= 

0.001753655 

F  (500) 

= 

-0.000985613 

F 

(548) 

= 

0.001401494 

F  (501) 

= 

-0.000667841 

F 

(549) 

= 

0.00108042 

F  (502) 

= 

-0.000295289 

F 

(550) 

= 

0.000795879 

F  (503) 

= 

0.000106065 

F 

(551) 

= 

0.000551021 

F  (504 ) 

= 

0.00050806 

F 

(552) 

= 

0.000346882 

F  (505) 

= 

0.00088232 

F 

(553) 

= 

0.000182649 

F  (506) 

= 

0.001202208 

F 

(554) 

= 

5 . 59829E-05 

F  (507) 

= 

0.001444645 

F 

(555) 

= 

-3 . 66224E-05 

F  (508) 

= 

0.001591677 

F 

(556) 

= 

-9. 94761E-05 

F  (509) 

= 

0.001631677 

F 

(557) 

= 

-0.000137334 

F (510) 

= 

0.001560107 

F 

(558) 

= 

-0.000155078 

F  (511 ) 

= 

0.001379813 

F 

(559) 

= 

-0.000157442 

F (512) 

= 

0.001100825 

F 

(560) 

= 

-0.000148802 

F  (513) 

= 

0.00073969 

F 

(561) 

= 

-0.000133018 

F  (514 ) 

= 

0.000318405 

F 

(562) 

= 

-0.000113329 

F (515) 

= 

-0.000136994 

F 

(563) 

= 

-9 . 23109E-05 

F (516) 

= 

-0.000598062 

F 

(564) 

= 

-7 . 18891E-05 

F (517) 

= 

-0.001035706 

F 

(565) 

= 

-5 . 3387  6E-05 

F  (518) 

= 

-0.001421941 

F 

(566) 

= 

-3 . 75925E-05 

F  (519) 

= 

-0.001731552 

F 

(567) 

= 

-2 . 48133E-05 

F  (520) 

= 

-0.001943538 

F 

(568) 

= 

-1 . 50752E-05 

F  (521) 

= 

-0.002042278 

F 

(569) 

= 

-1 . 05419E-05 

F  (522) 

= 

-0.002018327 

F  (523) 

= 

-0.001868843 

F  (524 ) 

= 

-0.0015976 

F  (525) 

= 

-0.00121462 

F  (526) 

= 

-0.000735468 

F  (527) 

= 

-0.000180249 

18.  (continued). 
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